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Abstract 

A major impediment towards the industrial adoption of decentralized distributed systems comes from the difficulty to 
theoretically prove that these systems exhibit the required behavior. In this paper, we use probability theory to analyze 
a decentralized auto-scaling algorithm in which each node probabilistically decides to scale in or out. We prove that, 
in the context of dynamic workloads, the average load of the system is maintained within a variation interval with a 
given probability, provided that the number of nodes and the variation interval length are higher than certain bounds. 
The paper also proposes numerical algorithms for approximating these minimum bounds. 
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1. Introduction 

Scalability has always been a critical non-functional requirement for parallel and distributed systems that receive 
a variable workload, but with the advent of cloud computing, scalability is no longer bounded by the amount of 
physical resources that were initially allocated to the system. The on-demand provisioning of virtualized computing 
nodes in the cloud allows providers of Internet services to rent only the amount of resources needed to serve the 
current workload and then scale in and out to cope with the request rate variations 1 1 1. 

The dynamic and unpredictable nature of the workload experienced by many Web applications, which may sud- 
denly become popular or loose most of their users in favor of other more trendy applications, requires autonomic 
scaling mechanisms. Therefore, the autonomic provisioning of virtualized resources has emerged as a rich research 
direction leading to the proposal of many auto-scaling techniques. However, most of these techniques are executed 
by a central manager which, despite being able to apply advanced optimization algorithms, acts as a scalability bot- 
tleneck and introduces a single point of failure (see, for example, l2l[3]|4l). The scalability limitations of centralized 
management are very well described by Meng et al. in |51 based on their experience with VMware. 

P2P technologies have proved their effectiveness in building Internet applications that are both massively scalable 
and fault tolerant |6|. With the advent of large cloud data centers offering on demand access to computing resources, 
we argue that P2P overlay networks represent a viable solution for building elastic service systems that are capable to 
adapt their resource consumption to the dynamic workload. However, the shift from an Intemet-based P2P environ- 
ment to a cloud-based P2P environment requires a change in the way the system scales. Thus, in Internet-based P2P 
systems the peers join or leave the system at their will, while in a cloud environment the system itself should decide 
to scale in or out. 

To overcome the limitations of centralized management, we proposed a decentralized probabilistic auto-scaling 
(DEPAS) algorithm in which each node decides to add a new node, remove itself, or do nothing in a probabilistic 
manner. The nodes self-organize to form an unstructured overlay network where, through gossiping, each node is able 
to estimate the average load of the system. Therefore, facing a variable workload, the system is capable to resize itself 
with the aim of keeping the average load close to a given threshold, called desired load. The difficulty of decentralized 
systems to maintain a fixed value for a global property was overcome by allowing the average load to vary in an 
interval (called load variation interval) centered in the desired load. The presentation of the DEPAS algorithm was 
the subject of another paper fTj where we showed through extensive simulations of up to ten thousand nodes that the 
number of allocated nodes is close to the optimal one while the algorithm is highly scalable and robust. In this paper, 
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we use probability theory to analyze the behavior of the algorithm and provide the potential customer of DEPAS with 
methods for tuning its parameters in such a way that a certain level of trust can be guaranteed. 

Actually, one of the main challenges towards the widespread use of decentralized systems in the industry comes 
from the difficulty to guarantee that the system will exhibit the expected behavior. Moreover, the achievement of 
a global property usually depends on the proper configuration of some system parameters. Finding such a proper 
configuration represents the tricky part of designing a decentralized system. For the DEPAS algorithm the global 
property to be maintained is the average load of the system and the main configuration parameter is the length of the 
load variation interval. By keeping the average load within the variation interval the system avoids oscillations (i.e., 
additions and removals mixed in a row) and the total number of nodes stays close to the optimal value. 

Due to the randomized and decentralized nature of our algorithm it is not possible to guarantee that a global 
property is achieved in absolute terms. Therefore, we adopt a probabilistic approach and prove that the algorithm keeps 
the average load within the variation interval with a certain probability, called correctness probability. Intuitively, 
given the length of the load variation interval, the correctness probability increases with the increase in the number 
of nodes. Reversely, for a certain number of nodes, the correctness probability grows with the increase of the load 
variation interval. Thus, in both scenarios, we are interested if there are some minimum bounds so that a minimum 
correctness probability can be guaranteed for any configuration in which the number of nodes and the interval length 
are higher than the minimum bounds, respectively. 

To address the above problem, in this paper, the DEPAS algorithm is modeled as a set of Bernoulli trials. In this 
way, using probability theory, we formally prove that: (i) given the desired load, the load variation interval, and a 
minimum correctness probability, there is a minimum number of nodes so that the correctness probability is higher 
than the given threshold for any actual load and for any number of nodes higher than the given minimum; (ii) given the 
desired load, the number of nodes and a minimum correctness probability, there is a minimum interval length so that 
the correctness probability is higher than the minimum threshold for any actual load and for any interval length higher 
than the minimum. Additionally, from the formal analysis of DEPAS, we derive numerical algorithms for computing 
upper bounds on the minimum thresholds (based on the Chernoff-Hoeffding bounds) and compare these upper bounds 
with an estimation of the real minimums (based on the binomial formula for Bernoulli trials). The algorithms for 
both scenarios can be used to tune DEPAS at design time. Moreover, the algorithm corresponding to the second 
scenario can be also used at runtime to dynamically adjust the variation interval length of each node according to the 
continuously changing size of the system. 

The remaining of this paper is organized as follows. Section|2]describes the DEPAS algorithm and formulates the 
problem to be solved. The problem is formally analyzed and a proof is given in Section[3] The algorithms for tuning 
DEPAS are described in Section|4] while Section|5]discusses the tuning algorithms in the light of some experimental 
results. Related work is shown in Section|6] Finally, Section]?] concludes the paper. 

2. Decentralized probabilistic auto-scaling (DEPAS) 

The DEPAS algorithm was described in as part of a middleware for deploying massively scalable services in a 
cloud infrastructure. In this section, the algorithm is introduced with an emphasis on the problem of finding a subset 
of the parameter configurations for which a minimum correctness probability can be guaranteed. 

We assume a system composed of a set of computing nodes. A node can be either a physical machine or a virtual 
machine allocated from a cloud provider on a utility basis. The nodes are homogenous in the sense that they have 
the same hardware configuration and run the same piece of software comprising both the functional service and the 
components of our middleware: overlay management, load balancing, and auto-scaling. The overlay management 
algorithm organizes the nodes into an unstructured overlay network where each node has a fixed degree and a low 
standard deviation in-degree. This is an adapted version of the gossip-based protocol developed by Jelasity et al. [81. 
For load balancing we use a combined approach: a DNS is employed to assign the address of a node to each client 
while an internal decentralized load balancing protocol (such as the dimension exchange protocol Q) moves requests 
between neighbors in order to equalize the load across nodes. 

In the following, the term node will be used to denote both the machine and the service instance running on that 
machine. Thus, the main parameters of a node are the capacity and the load. The capacity of a node is the maximum 
number of requests per second that can be processed by the service deployed on that node and is derived through 
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Table 1: DEPAS notations 



n 


Number of nodes of the system 


Lo 


Desired load threshold (percent with respect to node capacity) 


L 


Average load of the system (percent with respect to node capacity) 


6 


Defines the allowed load variation 


pi 


Probability indicator: used to compute the node probability, p 


P 


Node-level probability used to make node addition/removal decisions 


Pa 


Minimum correctness probability threshold 



benchmarking. As the nodes are homogenous, they have the same capacity. The load of a node is computed at a given 
moment in time, as the percentage of the average number of requests per second that were scheduled on that node 
over a certain timeframe with respect to the capacity of the node. Then, in a homogenous system, the average load of 
the system is simply the arithmetic average of the loads of all nodes. Note that in the case when the workload received 
by the system overcomes its capacity, the average load is supra-unitary. 

Under these considerations, the goal of the auto-scaling algorithm is to maintain the average load of the system, 
noted with L, within a given interval, (Lq - S,Lq + S), where Lq is called desired load and S defines the allowed 
variation of the load (see Table [T] for a complete list of notations). To achieve this goal each node can execute two 
types of actions: remove itself and allocate one or more other nodes. 

The DEPAS algorithm is shown in listing [T] It is periodically run by each node and begins by retrieving an estima- 
tion of the average load of the system. Note that the average load is not computed at this time, but just retrieved from 
the component running the average protocol. Although there are gossip-based solutions for computing the average 
load of the system at each node |10|, in our experiments l?], for simplicity and higher scalability, we approximated 
the average load of the system by the average load of the node and its neighbors. If the load is less than or equal to 
L() - S, then the node computes a probability indicator using Eq. ([TJ and, because the indicator is sub-unitary in this 
case, the node uses it as the probability to remove itself. Otherwise, if the load is higher than or equal to Lq + 6, then 
the probability indicator is computed using the same equation. In this situation, the indicator can be supra-unitary 
where its integer part represents the number of nodes to be added for sure, while its fractional part is used as the 
probability to add another node. Note that the random() function generates a uniformly distributed random decimal 
number between and 1 . 

Pi = — 7 (1) 

The desired load, Lq, is subject of a tradeoff. On one hand, a high value reduces the number of nodes, but, in case 
of a sudden workload increase, leads to a severe degradation of the system performance before the system has the 
chance to allocate new nodes. On the other hand, a low desired load increases the tolerance of the system to sudden 
traffic bursts at the cost of allocating many nodes. As the desired load has a high impact on both performance and 
cost, the customer of DEPAS is in the best position to set its value. 

However, while the meaning and impact of the desired load is straightforward for the customer, this is not the case 
when it comes to the load variation threshold, 6. Actually, even though the customer should not need to care about 
6, they are definitely interested in two aspects of the algorithm, namely correctness and accuracy, that are directly 
impacted by 6. By correctness we understand the ability of the system to make a right provisioning decision in the 
first place, thus avoiding oscillations (i.e., additions and removals mixed in a row). Due to the randomized nature 
of DEPAS it is not possible to evaluate its correctness in absolute terms (i.e., correct or incorrect). Therefore, we 
introduce the notion of correctness probability and allow the customer to define the correctness of the algorithm by 
specifying a minimum threshold of the correctness probability (see Definition [TJ. Then, the accuracy of a correct 
algorithm is a measure of how close the allocated number of nodes is to the optimal one. Therefore, the accuracy 
of DEPAS is directly influenced by 6: the lower 6 is, the higher the accuracy is. In this paper, we provide a formal 
analysis of the link between correctness probability, load variation threshold, and number of nodes. Our goal is to 
provide the customer with an analytical method for configuring DEPAS so that it meets given correctness requirements 



3 



Algorithm 1 DEPAS 



while true do 
wait(time frame) 
L <— get Averages ysteniLoadi) 
if L < Lq- 6 then 

pi <— computeProbabilityIndicator(L, Lq) 

P ^ pi 

if /7 < randomQ then 

removes elf() 
end if 
else 

if L> Li) + d then 

pi «— compute? robabilitylndicatoriL, Lq) 
m <— \_pi\ 
P <- (pi) 

if p < randomQ then 

m <— OT + 1 
end if 

addNodes(m) 
end if 
end if 
end while 



with an as high as possible accuracy. 

Definition 1. Let Lq and 6 be the desired load and the load variation threshold of a DEPAS instance, respectively. 
Then, the correctness probability is the probability that, after DEPAS has been run by each node, the new average 
load is in the interval (Lq — 6,Lq + 6). The correctness probability is noted with P(Lq — 6 < L < Lq + 6). Consequently, 
we say that an instance of DEPAS is correct if its correctness probability is higher than or equal to a given threshold, 
denoted by Pq. 

More concretely, we consider two usage scenarios of the DEPAS algorithm. In both scenarios, the customer 
specifies the minimum correctness probability, Pq. In the first scenario - called Min n - the customer is able to predict 
the minimum workload of the system and implicitly the minimum number of nodes and they are interested in the 
values of 6 for which the correctness probability is higher than Pq. The second scenario - called Min 5 - is concerned 
with the case when the customer has set a small value for S in order to obtain a very accurate algorithm and wants to 
find out for which system sizes a correctness probability higher than Pq can be guaranteed. Both scenarios are applied 
at design time before deploying DEPAS in the production environment. However, we can imagine the Min 6 scenario 
being also used at runtime as a subroutine of DEPAS in order to dynamically adjust the value of 6 at each node, 
provided that an estimation of the system size is available at each node ifTOlfTTI . By applying the Min 6 scenario at 
runtime we expect to increase the accuracy of DEPAS. However, as opposed to the design-time version, a runtime Min 
6 algorithm, being executed at each node, must be fast and have a low resource consumption in order not to overload 
the system. 

In this paper, we address the above challenges for the simplified case when each node computes the same proba- 
bility indicator, pi. This implies that each node precisely estimates the average system load. We also assume that all 
nodes have synchronized clocks and simultaneously run the DEPAS algorithm, although this constraint is not needed 
in practice. In other words, the algorithm is assumed to work in cycles where, in each cycle, each node uses the same 
probability to decide upon the execution of a scaling action. 

Looking back at listing [T] we notice that if pi > 1, which happens when L > 2Lq, then n'ipi\ is the number of 
nodes that are added for sure (where n is the number of existing nodes), which represents the deterministic part of the 
decision. Therefore, they have no impact on the correctness probability and the case when L > 2Lq is reducible to 
the case when L e [Lq + 6,2 ■ Lq). On the other hand, the addition and removal cases are symmetric. Consequently, 
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for simplicity and without loss of generality, we will consider only the addition case. Under these considerations, the 
formula for computing the probability of each node, p, is given by Eq. (|2]i. 

^^L-Lo^ L€ [Lo + 5,2Lo). (2) 

M) 

We want to express the correctness probability in function of the number of added nodes. For this purpose. 
Theorem [T] defines the notion of optimal number of nodes to be added as a rational number. In Theorem |2] the 
correctness probability is expressed in function of the number of added nodes and the optimal number of nodes 
corresponding to the bounds of the load variation interval. 

Theorem 1. Let Lq € (0, 1), a system composed of n homogenous nodes, and L e [Lq + S,2Lo) the average load of 
the system. Then, the optimal number of nodes to be added to the system so that the new average load will be equal 
to Lq, denoted by M(n, L, Lq), is given by 

L-Lo 

Min,L,Lo) = n ^ . (3) 
Lq 

Proof. Let m be the number of nodes to be added and C the common capacity of all nodes. As the system has the 
same workload before and after provisioning the new nodes, we have nLC - {n + m)LoC, from where it turns out that 
m^n^. □ 

Theorem 2. Let Lq e (0, 1), a system composed of n homogenous nodes, L e [Lq + 6,2Lq) the average load of 
the system, and L' e (0, 2Lq) the average load of the system after the addition of m nodes. Then, the correctness 
probability is equal to the probability ofm e (M(n, L, Lq + 6), M(n, L, Lq — 6)), as expressed by the formula below: 

P{Lq -6<L' <Lq + 6)^ P(M(n, L, Lq + 6) < m < M(n, L, Lq - 6)) . 

Proof. As the workload of the system remains unchanged, we have nL - {n + m)L', which implies that L' - 

Then, P{Lq-6 < L' < Lq + 6) = P{Lq-6 < J^L- < Lq+6) ^ > ^ > . _„ < m < ^ -„) = 

P(n^^j^ <m< n^T^) = P{M{n, L, Lq + 5) < m < M{n, L, Lq - 6)) (by applying Theorem[l|. □ 

In this section, we described the DEPAS algorithm and the problem we want to solve: finding whether and in 
which conditions a minimum correctness probability can be guaranteed. This section also prepared the ground for the 
formalization and theoretical analysis of the problem, which falls within the scope of the next section. 



3. Theoretical analysis 

As stated in the previous section, the DEPAS algorithm is assumed to work in cycles. In each cycle, each node 
uses the same probability to decide whether to add a new node or not. Consequently, a cycle of the algorithm can be 
modeled as a set of n Bernoulli trials, Xi,X2, ■.■,X„. We denote by 5„ the outcome of the experiment, which is the 
number of nodes added in the respective cycle. Under this formalization. Theorem [3] proves that the formula for p 
described by Eq. (j2]) was correctly chosen. 

Theorem 3. Let us consider a cycle of the DEPAS algorithm where each node adds a new node with probability 
p — (L — Lq)/Lq. Then, the expected number of added nodes is equal to the optimal number of nodes to be added. 

Proof. If S„ is the number of added nodes, then taking into consideration that S„ has a binomial distribution of 
parameters n and p it follows that E(S„) - np - M{n, L, Lq) (according to Theorem[T]i. □ 

Moreover, the correctness probability can be computed by using the binomial probability distribution: 

lM(n,L,Lo-S)\ , 

P(M(n, L, Lo + 6)< m < M(n, L, Lq -6))^ ^ P p'(l - p)"'' (4) 

i=\M(n.L,U)+5)'\ 

The problem with the expression from the left-hand side of Eq. (|4]) is that it is discontinuous and non-monotonic 
with respect to L and non-monotonic with respect to n and 5, thus increasing the risk of errors when using it with 
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numerical optimization algorithms. Moreover, its evaluation is expensive due to the binomial coefficients. Therefore, 
a faster and less error-prone method is needed for estimating the correctness probability, or at least its lower bound. 

Theorem |4] uses the Chernoff'-Hoeffding bounds [12J to compute a lower bound of the correctness probability. 
More precisely, it proves that for any probability threshold Pq € (0, 1), there is a minimum 6* (for a fixed n) or a 
minimum n* (for a fixed 6) so that the correctness probability is higher than or equal to Pq for all 6 > 6* or for all 
n>n*, respectively. The proof of the theorem also provides a method for computing 6* and n* . 

Theorem 4. Let us consider the DEPAS algorithm under the above assumptions. A cycle of the algorithm is modeled 
as a set ofn Bernoulli trials. Then, the following two affirmations hold. 

a. For any Pq € (0, 1) and 6 — (5o, there exists n* — n*(Po, do) G Z+ such that 

P (Min, L, Lo +6q)<S„< M{n, L, Lq - 6o)) > Po, V L e (Lq + 6, 2Lo), ^ n > n* . (5) 



b. For any Pq e (0, 1) and « = no £ 2+ satisfying 



ir 2 



"0 



'-[-3j -[-3j 
there exists 6* — <5*(fo,«o) 6 (0, 1) such that 

P {Mino, L, Lo + 6)< S„„ < M{no, L, Lq - 6)) > Pq, V L e (Lq + 6, 2Lq), \/ 6 > d* . (7) 

Proof. To simplify this problem, we will use the notations (re-scaling) 

L-Lq 5 

= /? e (0, 1) and — = e e (0, 1). 

Lq Lq 

Denoting 

Probip, s,n)^P (M(n, L,Lq + 6) < S„ < M(n, L, Lq - 5)) 

IL-Lq-6 S„ L-Lq + 6 

- P\ < — < 

\ Lq+5 

1 11 — P. s . 

= p 



\ Lq + 6 n Lq - 5 
\l+e n 



our problem is twofold: 

a. For a fixed s - sq, estimate the smallest value of n e Z+ such that Prob(p, sq, n) > Pq, for any p > sq; 

b. For a fixed n - no, estimate the smallest value of e e (0, 1) such that Prob{p, e, no) > Po, for any p > s. 
It can be easily seen that 

(S„ p- E\ (S„ p + e\ 



Prob{p,s,n) = 1 - < > /ilf). 

\n \ + e I \ n I - si 



p — s 

First, as < < p, the theorem of Chernoff-Hoeff'ding bounds provides the following inequality: 

1 -I- e 



\ n 1 + si 



where 

X 1 — X 

D[x,y] - xln — h ( 1 - x) In 

y l-y 
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represents the Kullback-Leibler divergence between Bernoulli distributed random variables with parameters x and y 
respectively, x,y e (0, 1). 

On the other hand, we notice that if p > 1 - 2s, we have > 1 and hence, p{— > | = 0. However, if 



p < I - 2s, we have p < 



p + s 
l-s 



< 1 and the theorem of Chernoff-Hoeffding bounds provides 



and 



(8) 



(9) 



\n ~ l-sl~ 

Therefore, we obtain the following lower bounds for the probability Prob{p, s, n): 

Prob(p,s,n) > 1 -e-«^[S.p] - e-"'^i'S-p] , M p & [s,\-2s], ee |o, ^j, « eZ+ 

Prob{p,s,n) > 1 - e'"'^^'Si'P]^ \/ p > max{e, 1 - 2s}, e e (0, 1), n e Z+ . 
In the following, let Bi(p, s, n) be the function from the right hand side of the inequality ([8]) 

Bi{p,s,n) = 1 -e-"4e./'] V e [e, 1 - 2fi], ee Jo, ^j, « eZ+ 

and Blip, s, n) the function from the right hand side of the inequality Q 

B2(p,s,n) = 1 - e-"^[f;|./'], V;, > max{e, 1 - 2s}, s e (0, 1), « e Z+ 
Both functions Bi and B2 are continuously difFerentiable on their respective domains of definition. It is easy to see 



that 



dBj 
dn 



■(p,s,n) — D 



1 +e' 



+Z) 



p + B 



1-e 



because of the positiveness of the Kullback-Leibler divergence, and hence, the function Bi is strictly increasing with 
respect to n. 

On the other hand. 



dBi ^ ^ d I \p-s 1\ ,,nr'' " „i d I \p + s ]\ 



where 



d_ 

ds' 

d_ 
ds' 



>0 



'^P finfi + f|_in|i_ 



2s 



>0 



for any s e \ Q, - j and p e[s,\ - 2s]. Therefore, the function B\ is strictly increasing with respect to s as well. 

In a similar way, it follows that B2 is also strictly increasing with respect to the variables s and n. 
Case a. When s - eq is fixed (i.e. 6 - 60 - sqLo), we have two subcases. 

a.l. If £() 6 (0, - I it can be proved that for any p e [sq, 1 - 2eo], there exists a unique n\{p) e (0, 00) satisfying the 



equation 



So, «!(/?)) = Pq. 



Indeed, for p e [so,! - 2eo] arbitrarily fixed, the function n i-> Bi(p,so,n) is continuous, strictly increasing. 
Blip, sq, 0) = -1 and lim Bi{p, sq, n) - 1. Therefore, n i-> Bi{p,SQ, n) is a bijective mapping between (0, 00) and the 



interval (-1, 1). As Pq £ (0, 1), there exists a unique solution ni(p) e (0, oo) of the equation Bi(p, sq, n) - Pq. More- 
over, the implicit function theorem guarantees that the function ni(p) defined above is continuously differentiable on 
[sq, 1 - 2e()], and hence, it is bounded. We denote 

n* = sup ni(p). 

pe[£o,I-2£o] 

On the other hand, for any p e (l - 2eo, 1), there exists a unique n2(p) e (0, oo) satisfying the equation 

B2(p, Sq, n2(p)) = Pq, 

given explicitly by 



niip) 



•ln(l -Po) 



As Um D 



P-SQ 

1 +eo" 



= oo, we obtain that n2{p) is bounded on the interval (1 - 2eo, 1) and let 
n* = sup n2(p). 

pe(l-2eo,l) 



Considering 



n* = n*(Po,so) - |^max{n*,n2 )] ^ Z+ 
from the fact that Z?i is increasing with respect to n, we clearly have 

Bi(p,so,n) > Bi(p,so,n*) > Bi(p,SQ,n*) > Bi(p,eQ,ni(p)) = Pq, Wpe [£(,,1 -2eo], V« > «*, 

and therefore, 

Prob(p, eq, n) > Pq, V p e [sq, 1 - 2sq\, Vn>n*. 
Similarly, as B2 is increasing with respect to n, we obtain 

B2(p, BQ, n) > B2(p, SQ, n*) > B2(p, bq, «J) > B2(p, cq, n2(p)) ^ Pq, V /? e (1 - 2eo, 1), Vn > n*, 

and therefore. 



In conclusion, we obtain: 



Probip, EQ, n) > Pq, V /:> € (1 - 2sq, 1), 'in>n*. 
Probip, Sq, n)> Pq, V p > sq, 'in>n*. 



a.2. If fio ^ 3 ' know from inequality (9 1 that 

Probip, SQ,n) ^ B2{p,eo,n), V p > sq, n e Z+. 
For any p e [eq, 1), the unique solution of the equation B2ip, cq, n) - Pq is n2(f ) given by ( [TO) i. Denoting 



n* = n*(Po,eo) = 



sup n2{p) 
pe[£o,l) 



as B2 is increasing with respect to n, we obtain 

B2ip, so, n) > B2(p, SQ, n*) > B2(p, sq, naCf )) = ^"0, V p e [eq, 1), Vn > n*, 

and hence 

Probip, Sq, n)> Pq, V /? > sq, Vn > n*. 
8 



(10) 



Case b. When « = «o e Z+ is fixed, we will consider two scenarios. 

b.l For P ^ 2 '■^^ lower bound for the probability Probip, s, no) is given by B[(p, s, no) according to (|8|) 
Let /7 e (o, be arbitrarily fixed. The equation 

Blip, E, no) = -Po 

has at most one solution e{p) < p. Indeed, we have Bi(p, 0, no) = -1 and 

limBi(/7,e,no) =!-(!- p)"" - e-"°4i^'H 
It is a simple calculus exercise to show that the function 

hi(p)= 1 -(1 -;,)"o _e-"o4i^>H 
is strictly increasing on the interval (0, 4), lim hi(p) - -1 and 



lim hi(p) - 1 



according to inequality (jiji. Tt follows that there exists a unique p* e ^0, |) such that hi{p*) = Pq. Therefore, if 

p e (0,p*], the equation Bi{p,e,no) - Po has no solution, but if p e (^p*, the equation B\{p,e,nQ) - Pq has a 
unique solution e\{p), satisfying e\{p) < p. Moreover, from the implicit function theorem we obtain that e\(p) is 
continuously differentiable on {j>*, and therefore, it is bounded. 

Note that lim Ei{p) - p* because we have lim Bi(p* ,e,no) - h\ip*) - Pq, and hence, we can extend the 

function ei by continuity, considering ei(p*) - p*. 
Denoting 

s* = sup Slip), 

we first observe that > ei(pt) = p'^ and therefore, using the fact that Bi is increasing with respect to e, we obtain 



Blip, e*, no) > Bi(p,ei(p),no) ^ Pq, V /? G 



b.2 For P ^ 2' lower bound for Prob{p, s, no) can be expressed from ( 8 i and ( 9 1 as 



B(p,E,no) 



The function B is increasing with respect to s 



Bi(p,e,no), ifee(0,^) 
B2(p,e,no), ifse^^-^,p'j 



and 



lim B(p,s, no) - limBi(/?,e, no) = -1 



Um B(p, s, no) = Um B2(p, e, no) = 1 - (1 - p)"" > 1 - 

e— >p e— >/7 

i-p. 



>Po 



The function B has a jump discontinuity at e = 



(11) 



lim B{p,s,no) = lim Bi{p,s,no) = 1 - e"'"'4TT'H - p"" = h2(p), 

Ej^ E^^ 



lim B(p,s,no) = lim B2(p,s,no) = 1 - e"'"""!*^-''] = h^{p). 

It is easy to check that the functions h2(p) and h^ip) defined above are continuous and strictly decreasing on the 
interval Q, l) and /i2(p) < h^ip) for any p e (|, l)- Moreover: 

Iim/Z2(p)= 1-1-1 -1-1 >A) and lim hiip) ^ -I, 



and 



Hm /i3(p) - 1 



> f and Hm hi{p) - 0. 



Therefore, there exist unique values p*, p* e l), p* < p*, such that h2{p*) - Pq and h^ip*) - Pq- 
If p e Q,/?*), it follows that 

lim B(p,s,nQ) = lim Bi(p,s,no) = h2{p) > h2{pl) = /"o 

and hence, since B\ is increasing with respect to e, the equation Bi{p,s,no) = Pq has a unique solution S2(p), 
satisfying E2(p) < ^ < 5- Moreover, from the implicit function theorem we obtain that S2(p) is continuously 

differentiable on (5, /J*)' and therefore, it is bounded. We can extended £2 by continuity by considering e2(/'2 ) - ~T^- 
Denoting 

e* = sup £2ip), 
and using the fact that B is increasing with respect to s, we obtain 



Bip,e^,no) > B(p,E2(p),no) = Bi(p,E2(p),no) = Pq, e (-,/?* 



(12) 



If p € (p* ,p*] we have /JsCp) > hj,(p*) - Pq and e* > S2(p2) - —p- > Taking into consideration that B2 is 
increasing with respect to e, we obtain: 

B(p,e^,no)^ B2(p,s*,no)> \im B2(p,e,no) ^ hip) > Pq, '^pe(p*,p*]. (13) 
If p e ipt, 1), since h^ip) < h-i{pt) - Pq, we have that 



P3 



lim B2{p, B, no) = hiip) < Po and lim B2(p, s, no) > Pq 

and therefore, the equation B2{p,s,no) - Pq has a unique solution £3(7?) such that ^ < e3(p) < p. The function 

e3(p) is continuously diff'erentiable, bounded, and it can be extended by continuity, considering s^ip*) - 2 
Denoting 

e* = sup S3(p), 

and using the fact that B is increasing with respect to e, we obtain 

B{p, E*, no) > B(p, s^ip), no) = B2(p, s^ip), no) = Pq, V /? € (/?*, l) . 



(14) 



Let 



e* = max{e*, • 



Combining the inequalities ( 11 1, (12 1, (13 1 and ( 14 1, and taking into consideration that B is increasing with respect to 
e, we obtain: 

Bip, e, no) > B(p, s* , no) > Po, V /? > e*, Ve > e*. 
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and finally: 



The proof is now complete. 



Probip, E, no) > Pq, S p> s* , Ve > s* . 



□ 



Remark 1. Instead of the Chernojf-Hoejfding inequalities used in the proof of the previous Theorem, it is possible to 
compute a lower bound of the correctness probability by means of the well known one-sided Chebyshev inequalities. 
For example, in a similar manner as in the previous proof, one may obtain the following formula for the estimate of 
n* , using Chebyshev bounds: 



max { 



Pod + si) + ^(l-e2)2+4p2g: 



.2^2 



8e2(l - Po) 



Pq (1 -2go)(l +eo)^ 
l-Po' 2£o(l - so)2 



I -Pi) 



1 - gQ 



if So < 



if So > 



(15) 



However, we note that Chernojf-Hoejfding inequalities lead to better results, since they give exponentially decreasing 
bounds on tail distributions, while Chebyshev inequalities yield only power-law bounds on tail decay. 



4. Algorithms for DEPAS tuning 

In this section, based on the results of the previous section, we formulate the algorithms for estimating the min- 
imum number of nodes and the minimum load variation threshold, respectively, for which the Chernoff-Hoeffding 
lower bounds of the correctness probability are higher than or equal to the given Pq. In order to get an idea of 
how close the estimations provided by the Chernoff-Hoeffding bounds are to the real minimums, we also provide 
algorithms for computing Min n and Min S based on the binomial formula. The functions used in the algorithms 
together with their properties are shown in Table |2] These properties (continuity, differentiability, monotony) deter- 
mine whether the functions can be used with some numerical optimization algorithms and their specific meaning was 
explained for each function in the proof of Theorem]?] 

4.1. Algorithms based on the Chernojf-Hoejfding bound 

Theorem ]4] used Chernoff-Hoeffding inequalities to derive a lower bound on the correctness probability. The 
bound is expressed by the Bi(p, s, n) and B2(p, s, n) functions (see Table]2]i. By fixing either s (in the Min n scenario) 
or n (in the Min 6 scenario), Bi and B2 become functions of two variables and one problem is to find the minimum 
value of the second variable (either n or e, respectively) so that the bound is higher than or equal to the given Pq, for 
any possible value of p. 



Algorithm 2 Min y monotonic 

Input: Function / : /i x /2 — > M continuously differentiable on the intervals /i and I2 and strictly increasing on I2, 
precisions si and S2 

Output: Minimum y* e Ii so that f(x,y*) > OVjc e h 

Let / be an empty list 

for all x' e /i increasing with step si do 

Use a root finding method to derive the root of the equation f{x',y) - with precision 52- 

Let y' be the found root. 

Append y' to / 
end for 
y* <— max(l) 



Algorithm]2]provides a solution to the above problem for a generic function of two variables / : /i x /2 — > M, that 
shares the same properties as Bi and B2- As in general, Bi and B2 are neither monotonic nor convex with respect to 
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Table 2: Functions used in the tuning algorithms 



Function 


Properties 


Bi{p,e,n) = I - e ""Li+s-pj _ g ""Vi-cfi 


Continuously difFerentiable, strictly increasing with respect to e and 
n when p G [e, 1 - 2e\, £ G (o, 5) , « G Z+ 


B2ip,s,n) = 1 - e-"^LT5i./'J 


Continuously difFerentiable, strictly increasing with respect to e and 
n when p > max{e, 1 - 2e), e g (0, 1), n g Z+ 


^ ^ -ln(l-Po) 


Continuously differentiable and concave when p g (1 - 2eo, 1) , so ^ 




Continuously difFerentiable, strictly increasing on (o, |) 


= 1 - (1 - p)"" - 






Continuously difFerentiable, strictly decreasing on l) 


/J3(P) = 1 - 


Continuously difFerentiable, strictly decreasing on 




Continuously difFerentiable, strictly increasing on (0, 00) 


LM(«X,Lo-<5)J 

Bin{L,6,n)= ^ rUl-;'r' 

(=rM(n,L,Lo+(5)l 


Discontinuous and non-monotonic with respect to L and non- 
monotonic with respect to 6 and n 
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p, the algorithm has to take increasing numbers from interval h with precision si. Then, taking advantage of the fact 
that B\ and B2 are strictly increasing with respect to e and n, for each considered value of y' e I2 we can use a root 
finding method to compute the minimum x' for which fix' ,y') > 0. Then, the maximum of the x' values satisfies the 
inequality for all y' values that were considered. 



Algorithm 3 Cernoff min n 

Input: Lo e (0, l),5o e (0, Lo),^'o ^ (0, 1), precisions s„ and Sp 
Output: «* e Z+ 



Compute with formula 15 
if So e (0, 5) then 

Use algorithm |2] to find the minimum n* e l..A^ for which f(p,n*) - B\{p,eQ,n*) - Pq > e 

[sq, 1 - 2fio] , = Sp, S2 = s„ 

Use a concave optimization method to find n* «— sup s(p) with precision Sp 

pe(l-2£o,I) 

n* = |^max{n*,n*)j 
end if 

if So e [i, 1) then 

Use a concave optimization method to find n* 



n* = \n*] 
end if 



sup s(p) with precision Sp 



Algorithm|2]is needed by both Algorithm |3] (i.e., Cernoff min ri) and Algorithm]?] (i.e., Cernoff min 5). In fact. 
Algorithms [3] and [4] just translate the proof of Theorem [4] into algorithmic language. Note that Algorithm [3] uses the 
Chebyshev inequality (which is weaker than the ChernofF-Hoeffding inequality) to compute an upper bound of the 
number of nodes. 



Algorithm 4 Cernoff min 6 

Input: Lo £ (0, l),no e Z+,Po £ (0, 1), precisions and Sp 
Output: 5* e (0, Lo) 
ifl-(i)""-(if <Pothen 

exit with no solution 
else 

Use a root finding method to derive the unique solution p* G (o, 5) of the equation h\{p) - Pq with precision Sp 
Use algorithm |2j to find the minimum s* < p* for which f(p,s*) = Bi(p,e*,no) - Pq > Q,'ip € [p*, 5] , ii - 

Use a root finding method to derive the unique solution p* e Q, 1^ of the equation h2(p) = Pq with precision Sp 
Use algorithm |2j to find the minimum e* < p* for which f{p,e*) - Bi(p,e*,no) - Pq > 0,'^p e (I,/?*] > ~ 

^pT ^2 ^£ 

Use a root finding method to derive the unique solution p* e Q, of the equation h3(p) = Pq with precision Sp 
Use algorithm jij to find the minimum s* < p* for which f{p, e*) - B2(p, £3 , «o) - ^"0 ^ 0, V/? e ^p*, ij ,si - 

Sp, S2 - Ss 

£* «— max|e*,e*,£3| 
6* <- e*Lo 
end if 



Note that, from a theoretical point of view, the accuracy of Algorithm [2] (and implicitly that of Algorithms [3] and 
|4| may be influenced by the precision si. The possible correlation between the output accuracy and the precision is 
checked experimentally in Section [5] 
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4.2. Algorithms based on the binomial formula 

The binomial formula, expressed by function Bin{L, 6, n) from Table|2] allows us to compute the exact value of the 
correctness probability. However, it is more difficult to compute the minimum n or 6 based on the binomial formula 
than based on the Chernoff-Hoeffding bounds because the binomial function Bin(L, 6, n) is discontinuous with respect 
to L and non-monotonic with respect to 6 and n. A solution is given by Algorithm|5] where x accounts for either 6 or 
n and y represents L. The main idea is to check for each y starting from its maximum possible value and decreasing 
by a given step whether the function is positive for all x taken with a certain precision. The solution is given by the 
minimum y for which the function is positive for all considered values of x. 



Algorithm 5 Min y 

1: Input: Function / : /i x /a — » M, precisions si and S2 

2: Output: Minimum y* e Ii so that f(x,y*) > OVx e Ii 

3: y ^ 

4: for all y & h decreasing with step S2 do 

5: / <- y 

6: for all X e Ii increasing with step s\ do 

7: if /(x,y) < Othen 

8: go to [12] 

9: end if 

10: end for 

11: end for 

12: y* <-y' + S2 



Concrete algorithms for approximating the minimum n (Binomial min n) and the minimum 5 (Binomial min 6) are 
obtained by instantiating Algorithm |5] for functions f\(x,y) - f\(L,n) - Bin(L,6o,n) - Pq and f2(x,y) - f2(L,d) - 
Bin(L, 6, no) - Pq, respectively. A maximum bound for n can be computed based on either Chebyshev or Chernoff- 
Hoeffding inequalities. 

Note that, due to the discontinuity of the binomial function, the binomial algorithms are prone to numerical errors 
and, therefore, not recommended to be used by the customer of DEPAS. Their sole objective is to allow us to estimate 
the accuracy of the ChemofF algorithms as shown in the next section. 

5. Experimental results 

The DEPAS tuning algorithms described in the previous section were implemented in Java. We used simple root 
and supremum finding algorithms based on the bisection method. The DEPAS Tuning Tool, which provides both 
graphical and command line facilities for running the tuning algorithms in a large spectrum of scenarios, is available 
for download fTTl . 

The DEPAS Tuning Tool was used to obtain the experimental results presented in this section. There are three 
objectives of the experiments: (i) to estimate the accuracy of Chernoff algorithms by comparing their results with the 
ones of the binomial algorithms, (ii) to make an idea about the execution time of the tuning algorithms, and (iii) to see 
how the accuracy of Chernoff algorithms is affected by the precision of the load. Two experiments (one with Min n 
algorithms and another one with Min 6 algorithms) were performed for each objective, thus resulting in a total of six 
experiments. 

All experiments were performed on Amazon EC2 Large Instances (7.5 GB memory, 4 EC2 Compute Units, 64-bit 
platform) running Amazon Linux. In all experiments we set Lq = 0.8, and Pq = 0.99. The default precisions were 
s„ = lO-\se = 10"^ and s,, = 10"^ 

In the first experiment, the number of nodes was computed with both Chernoff and binomial algorithms for several 
values of 6 taken from the interval [0.05,0.15] with step 0.005. From Figure|2] we can see that Chernoff min n and 
Binomial min n algorithms give relatively close results and that the results become closer with the increase of 6. For 
example, for S - 0.05 ChemoflF gives n - 342 and binomial gives n - 224, while for 6 = 0.15 the result of Chernoff 
is 41 compared to binomial's 30. 
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Figure 1: Min n experimental results: Chemoflf vs. Binomial 
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Figure 2: Min 6 experimental results: Chemoff vs. Binomial 



In the second experiment the values of 6 were computed with both Chernoff and binomial algorithms for several 
values of n taken from the interval [25, 1000] with step 5. As shown in Figure |2] the results are close and become 
closer with higher values of n. To give just two examples, for n - 100, Chernoff gives 5 - 0.094 and binomial gives 
6 = 0.075, while for n = 1000 Chernoff 's 5 is 0.03 and binomial's is 0.023. 

The third experiment was actually a series of 32 identical, but independently performed experiments that ran the 
Cernojfmin n and Binomial min n algorithms for several values of 5. The average execution time of both algorithms is 
shown in Table [3] Note that the result of the Cernoffmin n algorithm was used as an upper bound in the Binomial min 
n algorithm and, therefore, the execution time of the latter includes the execution time of the former We can notice 
that Cernoffmin n is very fast no matter what the value of 6 is, while Binomial min n is slow for small values of 6, but 
its execution time decreases with the increase of 6. 

The fourth experiment derived the execution time of the Cernoff min 5 and Binomial min 6 algorithms as an 
average of the execution times obtained from a set of 32 identical experiments. Several values of n were considered 
and the results are shown in Table [3] We can see that Chernoff performs extremely fast again, while Binomial is slow 
for high values of n, improving when n decreases. 

The last two experiments checked whether the results of Chernoff min n and Chernoff min 6 algorithms are influ- 
enced by the precision of the load, Sp. Therefore, experiments 5 and 6 executed the Chernoff algorithms in the same 
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Table 3: Average execution time (in seconds) for Min n and Min 6: Chemoff vs. Binomial 



Min n 


Min 6 


6 


Chernoff 


Binomial 


n 


Chemoff 


Binomial 


0.05 


0.013 


280.804 


25 


0.011 


1.580 


0.075 


0.010 


29.461 


250 


0.021 


29.004 


0.1 


0.008 


6.496 


500 


0.020 


89.767 


0.125 


0.006 


2.081 


750 


0.014 


178.924 


0.15 


0.005 


0.803 


1000 


0.016 


310.619 



conditions as experiments 1 and 2, respectively, but for three different load precisions: lO^^*, 10""*, 10"^. We found 
that the three-decimal results computed for the above load precisions were identical in all the considered cases, which 
means that it is worthless to use lower than lO"-' load precisions. 

In conclusion, the Chemoff algorithms produce results that are close to the real minimums, are very fast, and their 
results accuracy does not improve with low load precisions. Therefore, they can be used in both Min n and Min 6 
scenarios at both design time and runtime. 

6. Related work 

Randomized algorithms have found widespread applicability due to their simplicity and speed llT4ll . A brief 
survey on randomized algorithms lfT4l as well as a more recent and detailed one ifTSl were written by Motwani 
and Raghavan. The probabilistic analysis of randomized algorithms aims to provide probabilistic guarantees with 
respect to the likelihood of these algorithms to perform correctly or efficiently. For example, in the framework of 
resource allocation and admission control in transactional systems, Almeida et al. express the quality of service as the 
probability of executing every job within a maximum timeframe, and use the Markov and Chebyshev inequalities to 
compute an upper bound of this probability [2J. 

Particularly, randomization proved to be a powerful tool for building decentralized algorithms that ran in parallel 
and work with local information. Thus, probabilistic decentralized algorithms were proposed for solving the load 
balancing problem in large networks 11611171 . Theoretical analysis proved that, by using these algorithms, the expected 
or the maximum response time is less than a certain threshold with high probability. As for the Chernoff bound, it was 
applied in the probabilistic analysis of randomized packet routing algorithms for sparse networks and helped proving 
that certain algorithms are able to route all packets in a maximum number of steps with high probability [12, 18 J. 

A recent application of the Chernoff bound can be found in the decentralized and probabilistic solution to doc- 
ument clustering that was proposed by Papapetrou et al. |19l. In their approach, the correctness probability is the 
probability of assigning each document to the right cluster and - similarly to our case - is set by the user and used to 
tune the parameters of the algorithm. 

7. Conclusions 

The decentralized probabilistic auto-scaling (DEPAS) algorithm can be used to deploy elastic service systems that 
can quickly grow from tens to tens of thousands of computing nodes. However, the potential customer of DEPAS may 
not be convinced only by the experimental results that are, after all, dependent on the specific experimental scenarios 
being considered. 

Therefore, in this paper, we defined the correctness of DEPAS in a probabilistic manner and modeled DEPAS as 
a set of BernoulU trials. Then, the Chernoff-Hoeffding bounds were used to theoretically prove that there is a subset 
of configurations for which a minimum correctness probability can be guaranteed. Moreover, based on the theoretical 
results, we designed numerical algorithms for automatically tuning DEPAS so that it can be both correct and accurate. 
Through a set of experiments we showed that the results given by the Chernoff -based tuning algorithms are closed to 
the real minimums, which were estimated based on the binomial formula. 

In a future paper, the Chernoff min d algorithm will be used at runtime to dynamically adapt DEPAS, and the 
expected gain in accuracy will be experimentally checked for a set of realistic workload traces. 
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